Traffic Missing Data Completion With Spatial-temporal Correlations

1 The missing data problem remains as a difficulty in transportation information system, which seriously 2 restricted the application of intelligent transportation system, e,g. traffic control and traffic flow prediction. 3 To solve this problem, numerous imputation methods had been proposed in the last decade. However, few 4 existing studies had fully used the spatial correlation for traffic data imputation. In this paper, tensor based 5 imputing method, which had been proven to be an effective imputation method, is applied to multi-detector 6 missing data imputation for freeway corridor by constructing the traffic data into a 4-way spatial tensor. We 7 make three main contributions in this paper: (a) Various tensor patterns are explored to model the traffic 8 data, and take the multi-detectors into account. (b) Various tensor completion methods are explored and 9 evaluated for missing traffic data imputation. Experiments show HaLRTC is more robust for missing traffic 10 data than TDI. (c) The coefficient of the number of loop detectors used for missing traffic volume and speed 11 data imputation is studied. Experiment results show the number of locations related to the spatial-temporal 12 correlation of traffic data. 13 TRB 2014 Annual Meeting Paper revised from original submittal. Huachun Tan et.al. 2 INTRODUCTION 14 With a steady increase of freeway traffic in the recent years worldwide, the traffic congestion of freeways 15 becomes more serious. The freeway traffic congestion can no longer be dealt with simply by extending more 16 highways for economical and environmental reasons (Kerner, 2009). As a consequence, the optimization of 17 existing traffic network especially the freeway corridor control (Liu et al., 2011) has increasingly become a 18 more desirable alternative for management of freeway traffic congestion. Intelligent transportation systems 19 (ITS) play a significant role in optimizing the existing traffic network. Real-time traffic data is one of the 20 key factor to ITS. It is evidently indicated that the conventional ITS will eventually evolve into a data-driven 21 intelligent transportation system. And traffic data that are collected from multiple sources such as loop 22 detector, GPS and video sensors will become more and more important in ITS. (Ran et al., 2012; Zhang 23 et al., 2011) 24 Unfortunately, missing data problems are inevitable due to detector faults or transmission distortion 25 (Lin & Chang, 2006; Faouzi et al., 2011), which seriously restricts the application and generalization of 26 intelligent transportation systme. For example, the traffic control system requires sufficient traffic flow data 27 (i.e,traffic volumes, occupancy rates, and flow speeds) to generate appropriate traffic management strategies 28 (Carlson et al., 2010). In traffic forecast area, if there exists missing data, the predicting performance 29 will reduce sharply (Xu et al., 2010; Van Lint et al., 2005). Without proper imputation methods, traffic 30 counts with missing values are usually either discarded or simply estimated, which may seriously affect the 31 performance of ITS. Consequently, it is very urgent to develop a method with better effect to estimating the 32 missing data. 33 The frequently used imputation methods for missing traffic data are historical (neighboring) im34 putation methods(Ni et al., 2005), spline (including linear)/regression imputation methods (Chen & Shao, 35 2000), autoregressive integrated moving average (ARIMA) models (Zhong et al., 2004) and Probabilistic 36 Principal Component Analysis (Qu et al., 2009). These methods focus on imputing missing data for a sin37 gle loop detector, which often utilize the temporal correlations such as day mode periodicity, week mode 38 periodicity and interval variation of traffic data to estimate missing data. Nevertheless, the traffic data are 39 spatial-temporal correlated (Wu et al., 2012; Krawczyk et al., 2011) . Compared with temporal correlations, 40 the spatial correlations of traffic data have not been fully utilized. The most state-of-art methods only use 41 spatial information from neighbor detectors (Zhang, 2013; Zhang & Liu, 2009; Li et al., 2013). However, 42 the traffic data are correlated not only in short-distance (Liu et al., 2009b), but also in a large area (Min & 43 Wynter, 2011) especially in a freeway corridor (Van Lint & Hoogendoorn, 2010). As a result, only using 44 neighbor detector information is not the best approach for imputation of missing traffic data. 45 Recently, a tensor (multi-way array) based method (Tan et al., 2013b; Huachun Tan & Zhang, 2013) 46 has been applied to missing traffic data imputation and outlier traffic data recovery. The traffic data are mod47 eled by multi-way matrix (tensor) pattern, and the missing traffic data are estimated by tensor completion 48 method. Tensor completion allows for combining and utilizing the multi-mode temporal correlations (e.g., 49 week-mode, day-mode, and interval-mode) to estimate the missing data, which has been proved to be a effi50 cient tool to model traffic data for missing traffic data imputation. Despite the good results of tensor-based 51 method, this work is still applied for single loop detector missing data imputation. 52 In this paper, we focus on the missing traffic data completion for multi-loop detectors on freeway 53 corridor. Motivated by the power of tensor pattern in modeling multi-correlations of traffic data and the 54 reliable performance of tensor completion in missing traffic data imputation, this paper explores the ability 55 of tensor based method for multi-loop detector’s missing data imputation. The traffic data are constructed 56 into various 4-way spatial tensor, which covers the spatial information of the freeway corridor. Two tensor 57 completion methods, including HaLRTC (Liu et al., 2009a) and TDI (Tan et al., 2013b), are explored to 58 mine the underlying spatial-temporal information and impute the missing traffic data. Experimental results 59 on missing traffic volume and speed data show that the 4-way tensor considering the spatial information 60 is better than 3-way tensor without spatial correlation. Tensor completion method based on trace norm 61 TRB 2014 Annual Meeting Paper revised from original submittal. Huachun Tan et.al. 3 HaLRTC outperforms the method base on tensor decomposition TDI. The best number of loop detectors 62 for missing traffic data completion is also studied. Experiment results show the spatial-temporal correlation 63 of traffic data related to the number of loop detectors. 64 This paper is organized as follows: The necessary knowledge about tensor and tensor completion 65 are given in section 2. The tensor model for freeway corridor is conducted in section 3. In section 4, the 66 experiment results are given. The conclusion and future works are discussed in section 5. 67 TENSOR BASIC AND TENSOR COMPLETION 68 Notation and Tensor 69 Tensor which is also called the multidimensional array is the higher-order generalization of vector and 70 matrix. In this paper, the nomenclatures and the notations in (Acar et al., 2011; Tan et al., 2013a) on 71 tensor are partially adopted. Scalars are denoted by lowercase letters (a, b, c. . . ), vectors by bold lowercase 72 letters (a,b, c. . . ),and matrices by uppercase letters (A,B,C. . . ). Tensors are written as calligraphic letters 73 (A,B, C. . . ). 74 N-mode tensors are denoted as A ∈ RI1×I2×...×IN . Its elements are denoted as ai1...ik...in , where 75 1 ≤ ik ≤ Ik, 1 ≤ k ≤ N . The mode-n unfolding (also called matricization or flattening) of a tensor 76 A ∈ RI1×I2×...×IN is defined as unfold(A, n) = An. The tensor element (i1, i2, , iN ) is mapped to the 77 matrix element (in, j), where 78